TEDdet: Temporal Feature Exchange and Difference Network for Online Real-Time Action Detection

Localizing and interpreting human actions in videos require understanding the spatial temporal context of scenes. Aside from accurate detection, vast sensing scenarios real-world also mandate incremental, instantaneous processing scenes under restricted computational budgets. However, state-of-the-art detectors fail to meet above criteria. The main challenge lies their heavy architectural designs detection pipelines reason pertinent spatiotemporal information, such as incorporating 3D Convolutoinal Neural Networks (CNN) or extracting optical flow. With this insight, we propose a lightweight action tubelet detector coined TEDdet which unifies complementary feature aggregation motion modeling modules. Specifically, our Temporal Feature Exchange module induces interaction by adaptively aggregating 2D CNN features over successive frames. To address actors’ location shift sequence, Difference accumulates approximated pair-wise among target frames trajectory cues. These modules can be easily integrated with an existing anchor-free cooperatively model instances’ categories, sizes, movement for precise generation. exploits larger strides efficiently infer coarse-to-fine online manner. Without relying on flow models, demonstrates competitive accuracy at unprecedented speed (89 FPS) that is more compliant realistic applications. Codes will available https://github.com/alphadadajuju/TEDdet .